This invention relates to a data medium handling apparatus and a data medium handling method suitable for use, for example, for document handling in a financial organ, and more particularly to a data medium handling apparatus and a data medium handling method for performing handling of documents having various formats such as private documents.
In recent years, as apparatus which read character information as image data (an image) and then recognize characters, image data reading apparatus such as optical character reading apparatus (OCR apparatus) have been developed, and in various industries, the image data reading apparatus are used widely to achieve augmentation in efficiency in business and so forth.
For example, an operator who operates at the window of a financial organ or the like achieves augmentation in business by efficiently handling document media (documents) using an image data reading apparatus described above.
Particularly, in order to achieve augmentation in efficiency in business in which such document handling as described above is performed, it is required not only to handle document media of the same type (media for exclusive use for recognition processing) but also to automatically handle document media having various formats.
Thus, handling of a document medium using an image data reading apparatus can be performed for a plurality of kinds of document media by using, for example, such a document handling apparatus 1000 as shown in FIG. 115.
In particular, referring to FIG. 115, reference numeral 1001 denotes an image data reading apparatus for reading image data of a document. The image data reading apparatus 1001 is connected to a computer 1002 so that it can perform a reading operation of image data under the control of the computer 1002. It is to be noted that, for example, an image scanner or a facsimile is used as the image data reading apparatus 1001.
Meanwhile, the computer 1002 as a control apparatus for controlling the image data reading apparatus 1001 is composed of an inputting unit 1002-1 such as a keyboard or a mouse (only a keyboard is shown in FIG. 115) for inputting instructions, data and so forth from an operator, a computer mainframe 1002-2, and a display unit 1002-3 for displaying data, control information or the like. It is to be noted that character recognition processing of image data read by the image data reading apparatus 1001 is performed by the computer mainframe 1002-2.
Further, reference numeral hard disk 1003 denotes a hard disk. The hard disk 1003 is connected to the computer 1002 and stores in advance information (hereinafter referred to as xe2x80x9cdefinition object informationxe2x80x9d) 1003A to 1003C, . . . which designates, for each kind of document, for example, as seen in FIG. 116, position information of character data to be recognized, and a type, a number and so forth of characters.
It is to be noted that such an image data reading and recognition apparatus 1005 as shown in FIG. 116 or 117 may be used in place of the image data reading apparatus 1001 described above. The image data reading and recognition apparatus 1005 can perform a reading operation of image data similar to that of the image data reading apparatus 1001 as well as character recognition.
In order to recognize character data appearing on an electricity bill paid-up notice 1004 as shown, for example, in FIG. 116 using the image data reading and recognition apparatus 1005, a document kind (in this instance, the definition object information 1003B corresponding to the electricity bill paid-up notice 1004 (document B)) is designated by operation of the keyboard 1002-1 by the operator.
Then, the computer 1002 accesses the hard disk 1003 to extract the definition object information 1003B of the designated document and reports the definition object information 1003B to the image data reading and recognition apparatus 1005.
Consequently, the image data reading and recognition apparatus 1005 can effect reading of image data and character recognition processing based on the definition object information 1003B as control information from the computer 1002.
However, in the procedure of handling such a document as described hereinabove with reference to FIG. 116, since definition object information is designated for each document to be read by designation of the operator, a burden is imposed on the operator, and besides, as definition objects increase in number, a designation error may occur. Where it is required to handle several thousands kinds of documents, it is actually difficult for the operator to designate a definition object.
Therefore, also another method has been proposed wherein, as shown in FIG. 117, an ID number (in this instance, xe2x80x980102xe2x80x99) for identification of a document from any other document is recorded at a prescribed position 1004a of the document 1004 so that reading processing of the document may be performed automatically without such necessity for designation of a document kind by the operator as described above.
With the method just described, when image data of a document are to be read by the image data reading and recognition apparatus 1005, character recognition can be effected by first recognizing the ID number recorded at the predetermined position 1004a and then using the definition object information (in this instance, 1003B) corresponding to the recognized ID number.
However, when image data are to be read, if the position at which a document or the like is set in an optical reading apparatus (for example, a document handling apparatus) such as the image data reading apparatus 1001 or the image data reading and recognition apparatus 1005 is displaced, then even if a document same as, for example, a document whose definition object information is stored in advance in the hard disk 1003 is read, since coordinates of an image of a region of character data, a region of a figure or the like from a reference point (physical origin) do not exhibit the same coordinates as those of the definition object information, it is determined in a document handling apparatus or the like that the layouts are not the same as each other.
In a document handling apparatus or the like, character recognition processing is not sometimes performed regularly unless character recognition of image data is performed after the layout of read image data and the layout of definition object information are adjusted to each other. Therefore, reference points of individual images are extracted and coordinates of the individual images from the reference points are compared with each other to effect discrimination of coincidence between the layouts.
Here, the following two methods are dominantly used for extraction of a reference point of image data. It is to be noted that, as a precondition, an object document for reading is a document printed in advance and the print position of the document on a form is managed with a high degree of accuracy.
According to the first method, when image data of a document are to be read by an image data reading apparatus by which, as shown in FIG. 118(a), a form end edge 1006 of the document to be read and a reading background 1007 can be identified from each other, the left upper corner position P of the form end edge 1006 is determined as a reference point.
According to the second method, when image data of a document are to be read by an image scanner or a facsimile, since a reading background 1009 and a form end edge 1008 read cannot be identified from each other as seen in FIG. 118(b), a reference mark R is printed on an object document form for reading in advance and this reference mark R is extracted from image data and determined as a reference point.
The second method is advantageous in that, since a reference point position is printed as the reference mark R in advance, even if the position in which the document is set in a reading apparatus is displaced, the reference point can be extracted stably.
However, in a document handling apparatus or the like, even if a reference point is extracted accurately by the method described above, if the reading direction of image data is not a correct direction, then the character describing direction of image data is not a correct direction, and consequently, character recognition processing cannot be performed.
If, for example, as shown in FIG. 119, a transfer slip (document C) 1010 on which information is described in a horizontal direction is read in a wrong direction (in a vertical direction in FIG. 119), image data of the transfer slip 1010 thus read are displayed in a wrong orientation as denoted by reference symbol 1010A on the display unit 1002-3.
Therefore, upon character recognition processing by a document handling apparatus or the like, the operator must visually observe the display unit 1002-3 to discriminate whether or not the reading direction of the document is correct and, when the reading direction of the document is not correct, input, from the keyboard 1002-1, an instruction to rotate read image data of the document by 90 degrees or 180 degrees so as to effect rotational correction processing of the image data so that the image data of the document may be displayed in a correct direction as denoted by reference symbol 1010B on the display unit 1002-3.
By the way, in a conventional remittance job to an account in a bank, an operator inputs information of an account number, a name, an amount of money and so forth described on a document which is a remittance request slip from the keyboard 1002-1 or the like of the document handling apparatus 1000. In this instance, the operator visually observes the document and the display unit 1002-3 alternately to perform an inputting operation.
However, where the document and the display unit 1002-3 are visually observed alternately to perform an inputting operation in this manner, since many movements of the operator""s eye are involved, there is a subject in that the visual discernibility is deteriorated and there is the possibility that the matching between an item of the document and an item on the screen may be mistaken.
Thus, a further document handling apparatus has been proposed wherein a document is read by the image data reading apparatus 1001 or the like and read image data themselves are displayed on the display unit 1002-3 together with a result of character recognition of the image data.
With the document handling apparatus, since information described on a document can be visually observed directly on the display unit 1002-3, movements of the eye are reduced, and errors in matching between a result of character recognition and image data upon confirmation of and amendment to the result of character recognition can be reduced.
Meanwhile, in recent years, a method wherein data processing is performed in a concentrated manner at a high speed by a server using a client-server system is dominantly employed in financial organs or the like.
For example, it is proposed to interconnect a client installed for each branch (business office) of a financial organ and a server installed at a district center by a dedicated line or an exchange line to construct a client-server system and perform document handling collectively by the server to achieve augmentation in efficiency in business.
Where document handling is performed collectively by a server in this manner, since the amount of data concentrated upon the server becomes a very much amount, it is impossible for an operator to perform document handling by the server. Consequently, a system wherein a server automatically performs document handling without intervention of an operator must be designed.
Thus, if document identification processing in which such a document on which an ID number is described as described hereinabove with reference to FIG. 117 is applied to the client-server system just described, then the server can automatically identify the kind of a document and perform reading processing of the document.
Further, also with the client of the client-server system, where image data themselves read by the image data reading apparatus are displayed on the display unit together with a result of character recognition of the image data, information described on a document can be visually observed directly on the display unit to reduce errors in matching between the result of character recognition and the image data upon confirmation of and amendment to the result of character recognition.
However, in the technique which handles such a document as described hereinabove with reference FIG. 117, document media which can be read are only document media produced for exclusive use for the document handling apparatus, and since ordinary documents used conventionally cannot be used, there is a subject in this instance that documents for exclusive use must be produced newly.
Meanwhile, in such a first reference point extraction method as described hereinabove with reference to FIG. 118(a), while the print position of a document on a form must be managed with a high degree of accuracy, where a document printed, for example, by a word processor or the like is involved, if the form is set manually, then the print position varies in many cases every time printing is performed. Therefore, in such an instance, there is a subject that it is not suitable to set the left upper corner position P of the form end edge 1006 as a reference point.
Further, in such a second reference point extraction method as described hereinabove with reference to FIG. 118(b), object documents for reading are documents for exclusive use on which the reference mark R is printed, and there is a subject that, where ordinary documents on which the reference mark R is not printed are involved, the reference point cannot be extracted.
Further, also with the technique adopted in a recognition technique for a document reader or the like wherein a particular point of a table in a document is used as a reference point, there is a subject in that, where the layout of an object document for recognition cannot be specified to some degree, the technique does not function effectively.
It is to be noted that, even when the same document as the document whose image has been read formerly is read again, the image read may not be found same as the former one because of dust, blurring or the like upon image reading, and in such a case, the same reference point may not sometimes be extracted.
Further, in order for a server to automatically perform such rotational correction processing of image data as described hereinabove with reference to FIG. 119, it is necessary to perform character recognition of image data actually read and then perform discrimination whether the reading direction of the document is correct or wrong depending upon whether or not character recognition is possible, and there is a subject in that this disturbs augmentation in efficiency in job very much.
Meanwhile, in such a method as described hereinabove wherein a result of character recognition of image data and the image data themselves are displayed on a display unit to perform confirmation of and amendment to the result of character recognition, since matching between the image data and a confirmation item is performed by visual observation, there is a subject in that, where the image data include a large number of items to be confirmed, erroneous recognition cannot still be avoided.
Further, where the image data cannot be displayed at a time on the display unit, the display screen must be scrolled in order to refer to rearward data, and in this instance, a required key must be depressed to perform a scrolling operation. Accordingly, there is a subject in that the operation is complicated.
The present invention has been made in view of such subjects as described above, and it is an object of the present invention to provide a data medium handling apparatus and a data medium handling method by which handling of documents which are ordinary documents used conventionally and having various formats like private slips or the like can be performed.
It is another object of the present invention to provide a data medium handling method wherein, even for a document printed on plain paper by a word processor or the like, extraction processing of a reference point from image data of the printed document can normally be performed stably and automatically without using such a form end edge, a reference mark or the like as in the conventional techniques.
It is a further object of the present invention to provide a data medium handling method by which rotational correction processing of image data read by an image data reading apparatus can be performed automatically.
It is a still further object of the present invention to provide a data medium handling apparatus and a data medium handling method wherein part of image data is emphatically displayed based on information defined in advance to raise the recognizability and allow smooth confirmation and amendment processing.
It is a yet further object of the present invention to provide a data medium handling apparatus and a data medium handling method wherein a character image in units of a character is extracted from image data in units of a document or in units of an item and the character image is edited to reduce the movements of the type and the objects of comparison and besides allow confirmation with a high degree of accuracy so that the labor and the time required for a correct/wrong confirmation operation can be reduced remarkably.
According to the present invention, a data medium handling apparatus for recognizing, based on an image read from a data medium on which information is described in an arbitrary format, the information, is characterized in that it comprises means for extracting characteristics unique to the data medium including the format from the read image data and specifying, from the characteristics, a position at which information to be recognized is present, and image recognition means for recognizing the image data at the position specified by the means to discriminate the information.
In particular, according to the present invention, a data medium handling apparatus for recognizing, based on an image read from a data medium on which information is described in an arbitrary format, the information, is characterized in that it comprises a layout analysis processing section for extracting characteristics of a layout of the read image to analyze a structure of the layout and representing the layout logically based on the extracted characteristics, a candidate layout storage section in which candidate layouts are stored, and a layout collation processing section for retrieving the candidate layouts from the candidate layout storage section using the characteristics of the layout extracted by the layout analysis processing section as restrictive retrieval terms to collate whether or not a coincident candidate layout is present.
Here, the data medium handling apparatus may further comprise an analysis pre-processing section for performing required pre-processing for the image, which makes an object of recognition, prior to the layout analysis processing by the layout analysis processing section.
Or, the layout analysis processing section may at least include a layout characteristic extraction section for extracting characteristics of the layout of the image, and a layout analysis section for analyzing a structure of the layout based on the characteristics of the layout extracted by the layout characteristic extraction section.
Further, the layout analysis processing section is constructed such that it first extracts characteristics of the layout and then extracts a logical origin of the layout, and the layout collation processing section is constructed such that it collates whether or not there is a coincident candidate layout with reference to the logical origin extracted by the layout analysis processing section.
Here, where the layout has a table structure formed from ruled lines, the layout characteristic extraction section includes a ruled line extraction section for extracting the ruled lines of the layout while the layout analysis section includes a table structure analysis section for analyzing the table structure formed from the ruled lines based on the ruled lines of the layout extracted by the layout characteristic extraction section. In the meantime, where the layout has a no ruled line table structure which does not include ruled lines, the layout characteristic extraction section includes an item data extraction section for extracting item data of the layout while the layout analysis section includes a table structure analysis section for analyzing the no ruled line table structure based on the item data of the layout extracted by the layout characteristic extraction section.
Further, the layout analysis processing section may be constructed such that it determines a caption region in the image, recognizes caption item information in the caption region and determines corresponding item data.
Or, the data medium handling apparatus may be constructed such that the candidate layout storage section is constructed so as to store information of compressed candidate layouts, and the layout collation processing section compression processes the characteristics of the layout extracted by the layout analysis processing section and retrieves the compressed candidate layouts from the candidate layout storage section using the compressed characteristics of the layout as restrictive retrieval terms to collate whether or not a coincident candidate layout is present.
Or else, the candidate layout storage section may be constructed so as to update the candidate layout information by learning.
In the data medium handling apparatus of the present invention having the construction described above, in the information present position specification step, characteristics unique to a data medium including a format are extracted from read image data, and from the characteristics, a position at which information to be recognized is present is specified, and in the image recognition step, the image at the position specified in the preceding step is recognized to discriminate the information thereby to effect recognition of the information based on the image read from the data medium on which the information is described in an arbitrary format.
In particular, in the data medium handling apparatus of the present invention, in the layout analysis processing step, characteristics of a layout of a read image are extracted to analyze a structure of the layout and the layout is represented logically based on the extracted characteristics. and in the layout collation processing step, candidate layouts stored in the candidate layout storage section are retrieved using the characteristics of the layout extracted in the layout analysis processing step as restrictive retrieval terms to collate whether or not a coincident candidate layout is present thereby to effect recognition of the information based on the image read from the data medium on which the information is described in an arbitrary format.
In the layout analysis processing step, characteristics of the layout may be extracted first and then a logical origin of the layout may be extracted, and in the layout collation processing step, the information may be discriminated with reference to the logical origin extracted in the layout analysis processing step.
Further, in the layout analysis processing step, it may be discriminated whether or not the layout of the read image is in a predetermined direction, and the read image data may be converted based on a result of the discrimination so that the layout may be in the predetermined direction. The conversion processing of the read image data may be such processing as to rotate the layout.
Here, where the layout has a table structure formed from ruled lines, in the layout analysis processing step, the ruled lines of the layout are extracted, and the table structure formed from the ruled lines is analyzed based on the ruled lines of the layout extracted. In the meantime, where the layout has a no ruled line table structure which does not include ruled lines, in the layout analysis processing step, item data of the layout are extracted, and the no ruled line table structure is analyzed based on the item data of the layout extracted.
Further, in the layout analysis processing step, a caption region in the image may be determined, and caption item information in the caption region may be confirmed and corresponding item data may be determined.
Where information of compressed candidate layouts is stored into the candidate layout storage section, in the layout collation processing step, the characteristics of the layout extracted in the layout analysis processing step are compression processed, and the compressed candidate layouts are retrieved from the candidate layout storage section using the characteristics of the layout compressed in the preceding step as restrictive retrieval terms, and then, based on a result of the retrieval, it is collated whether or not a coincident candidate layout is present.
Further, in the candidate layout information is updated by the candidate layout storage section by learning.
Accordingly, the data medium handling apparatus of the present invention is advantageous in that, since various documents or document images which are produced daily are automatically analyzed electronically, the efficiency in entry jobs which make use of conventional entry systems (OCR entry systems, data entry systems and so forth) can be raised remarkably.
Further, since layout log information is produced based on a result of layout recognition of an arbitrary document of any kind and is registered by learning into the candidate layout storage section, discrimination of a document having the same layout can thereafter be performed automatically, and consequently, the labor and the cost required for production of documents for exclusive use can be reduced. Further, since entry is enabled immediately after such learning, reduction in period in which a system based on the present apparatus is installed can be achieved.
Furthermore, since image data of a document are normally analysis processed, automatic recognition processing for documents other than expensive OCR documents for exclusive use with which the cutting error or the print error is little.
According to the present invention, a data medium handling method for recognizing, based on an image read from a data medium on which information is described in an arbitrary format, the information, is characterized in that it comprises the logical origin extraction step of extracting a logical origin of the format, the information present position specification step of extracting characteristics unique to the data medium including the format with reference to the logical origin extracted in the preceding step and specifying, from the characteristics, a position at which information to be recognized is present, and the image recognition step of discriminating, when the image at the position specified in the preceding step is to be recognized, the information with reference to the logical origin extracted in the logical origin extraction step.
In the logical origin extraction step, not only the logical origin of the format may be extracted, but also logical origin auxiliary coordinate information which provides auxiliary information to the logical origin maybe extracted, and in the image recognition step, the information may be discriminated with reference to the logical original and the logical origin auxiliary coordinate information.
Where the layout has a table structure formed from ruled lines, coordinate information regarding one of corners which define the table structure is extracted as the logical origin of the format.
In this instance, coordinate information regarding the remaining corners may be extracted as the logical origin auxiliary coordinate information.
Particularly, where the layout has a table structure formed from ruled lines, coordinate information of one of corners defining the table structure which has a minimum X coordinate and a minimum Y coordinate is extracted as the logical origin of the format.
In this instance, coordinate information of another one of the corners which has a maximum X coordinate and the minimum Y coordinate, a further one of the corners which has the minimum X coordinate and a maximum Y coordinate and a still further one of the corners which has the maximum X coordinate and the maximum Y coordinate may be extracted as the logical origin auxiliary coordinate information.
On the other hand, where the layout has a no ruled line table structure which does not include ruled lines, coordinate information regarding one of corners of a circumscribed rectangle of item data in the layout is extracted as the logical origin of the format.
In this instance, coordinate information regarding the remaining angles of the circumscribed rectangle may be extracted as the logical origin auxiliary coordinate information.
More particularly, where the layout has a no ruled line table structure which does not include ruled lines, coordinate information of one of corners of the circumscribed rectangle which has a minimum X coordinate and a minimum Y coordinate is extracted as the logical origin of the format.
In this instance, coordinate information of another one of the corners of the circumscribed rectangle which has a maximum X coordinate and the minimum Y coordinate, a further one of the corners which has the minimum X coordinate and a maximum Y coordinate and a still further one of the corners which has the maximum X coordinate and the maximum Y coordinate may be extracted as the logical origin auxiliary coordinate information.
Accordingly, the data medium handling method of the present invention is advantageous in that, since a layout analysis of a document of an object of processing is performed based on ruled line items or item data extracted from image data and predetermined coordinates from among coordinates of corners of a region obtained by the layout analysis are determined as logical origin coordinates and correction coordinates, logical origin coordinates and correction coordinates can be extracted even from image data of ordinary documents other than documents for exclusive use on which a reference mark or the like is printed.
Further, since correction coordinates are extracted, upon extraction of a logical origin, simultaneously as additional information to the logical origin, coincidence can be detected with certainty upon collation of logical origins in the layout collation processing.
In particular, since collation of logical origins is performed through collation of relative coordinates of correction coordinates with logical origin coordinates taken into consideration, even if positional displacement occurs upon image reading, collation processing in layout can be performed with certainty.
Further, according to the present invention, a data medium handling method for recognizing, based on an image read from a data medium on which information is described in an arbitrary format, the information, is characterized in that characteristics unique to the data medium including the format are extracted from the read image data and then compression processed, and reference compression information stored in a storage section is retrieved using the compression processed characteristics as restrictive retrieval terms to collate whether or not there is coincident reference compression information, whereafter the image at a specified position is recognized based on a result of the collation to discriminate the information.
Here, prior to the compression processing for the characteristics, obliqueness correction and distortion correction processing may be performed for the read image data.
Further, upon the compression processing for the characteristics, horizontal ruled lines which construct ruled line items of the read image data are referred to to perform Y coordinate unification processing for those of the ruled line items whose upper side ruled lines commonly have a same ruled line, and vertical ruled lines which construct the ruled line items of the read image data are referred to to perform X coordinate unification processing for those of the ruled line items whose left side ruled lines commonly have a same ruled line.
Or, upon the compression processing for the characteristics, the characteristics are converted into a corresponding bit stream, and the bit stream is compression processed and set as a matrix table.
Accordingly, the data medium processing method of the present invention is advantageous in that, since a layout of a ruled line item which is one of components of a layout is compression processed and represented as a matrix table and, upon collation of layouts, such matrix tables are collated with each other, the layout collation processing speed increases merely linearly in proportion to the number of layouts of an object of collation without relying upon the number of ruled line items, and collation processing of layouts of a large number of documents can be performed at a high speed.
Further, since, prior to compression processing of a bit table on which a layout of ruled line items is reflected, obliqueness correction and distortion correction processing is performed for the read image data and Y coordinate unification processing and X coordinate unification processing are performed for the ruled line items, a variation in layout which occurs upon reading of image data can be absorbed.
Further, according to the present invention, a data medium handling method for recognizing, based on an image read from a data medium on which information is described in an arbitrary format, the information, is characterized in that it comprises the registration step of extracting characteristics of a layout of the read image, analyzing a structure of the layout, displaying the analyzed layout structure on a display unit, setting a predetermined characteristic data recognition region for the displayed layout structure, inputting code data corresponding to emblem information in the region through an external inputting apparatus, producing region characteristic data from the characteristic data recognition region and the code data, and registering the region characteristic data in a linked relationship with the analyzed layout structure, and the collation step of extracting characteristics of an image read separately, analyzing a structure of the layout, retrieving the region characteristic data registered in the registration step and the layout structure as reference data and collating whether or not there is a coincident region characteristic data.
Here, when code data corresponding to the emblem information in the region are inputted in the registration step, code data having significance corresponding to the emblem information are inputted.
Accordingly, with the data medium processing method of the present invention, since functions of a region image characteristic extraction mechanism and a characteristic collation mechanism similar to those of a character recognition apparatus are expanded and combined with a layout recognition system, recognition processing of an arbitrary graphic image in a designated region can be performed in an interlocking relationship with layout recognition processing.
Further, since region characteristic data of an object of collation are recorded in an linked relationship with layout information, the object of collation can be restricted in units of a layout, and consequently, such collation processing of a large amount as required by ordinary character recognition processing need not be performed and a high recognition rate can be achieved.
Furthermore, if an item of a rubber seal, a signature or the like is designated as a characteristic data recognition item, then such an item as a rubber seal, a signature or the like which cannot be recognized by ordinary character recognition can be recognized, and corresponding code data can be acquired as a result of recognition of a characteristic region of an item of a rubber seal, a signature or the like.
Further, according to the present invention, a data medium handling method for recognizing, based on an image read from a data medium on which information is described in an arbitrary format, the information, is characterized in that it comprises the layout analysis processing step of extracting characteristics of a layout of the read image, analyzing a structure of the layout, determining a caption region in the image based on the extracted characteristics, recognizing caption item information in the caption region, determining corresponding item data and logically representing the layout, and the layout collation processing step of retrieving candidate layouts stored in a candidate layout storage section using the characteristics of the layout extracted in the layout analysis processing step as restrictive retrieval terms to collate whether or not there is a coincident candidate layout.
The data medium handling method is further characterized in that the layout analysis processing step includes the caption position candidate determination step of determining caption position candidates based on a result of the analysis of the structure of the layout, and the caption item collation step of collating the caption item information in the caption region with the caption position candidates determined in the preceding step.
Upon the collation of the caption item in the caption item collation step, the collation processing of the caption item is performed after normalization processing is performed for the image data to be collated.
Accordingly, the data medium handling method of the present invention is advantageous in that, since a caption item and item data corresponding to the caption item can be identified by the layout recognition processing by performing the caption analysis processing of retrieving and collating a caption position from a ruled line item structure of a document image of an object of analysis to automatically analyze item information corresponding to a caption described on a document, character recognition of item data can be performed immediately without performing character recognition processing of a caption item.
Meanwhile, a data medium handling apparatus according to the present invention is characterized in that it comprises an image data reading apparatus for reading image data of a document as a data medium having required information described in a format thereof, an image data storage memory for storing the image data of the document read by the image data reading apparatus, document identification information extraction means for extracting required document identification information described on the document from the image data of the document stored in the image data storage memory, a document identification dictionary for registering the document identification information extracted by the document identification information extraction means as document identification information for a particular document, reference means for referring to image data of an arbitrary document read by the image data reading apparatus and stored in the image data storage memory to detect whether or not the image data of the arbitrary document include the document identification information registered in the document identification dictionary, and document identification means for identifying, based on a result of the reference by the reference means, whether or not the arbitrary document is the particular document.
Further, a data medium handling apparatus according to the present invention is characterized in that it comprises an image data reading apparatus for reading image data of a document as a data medium having required information described in a format thereof, an image data storage memory and a file memory for storing the image data of the document read by the image data reading apparatus, document identification information extraction means for extracting required document identification information described on the document from the image data of the document stored in the image data storage memory, a document identification dictionary for registering the document identification information extracted by the document identification information extraction means as document identification information for a particular document, verification means for reading out the image data of the particular document stored in the file memory and verifying whether or not the image data of the particular document include the document identification information registered in the document identification dictionary, discrimination means for discriminating, based on a result of the verification by the verification means, whether or not recognition of the particular document is possible, reference means for referring to image data of an arbitrary document read by the image data reading apparatus and stored in the image data storage memory to detect whether or not the image data of the arbitrary document include the document identification information registered in the document identification dictionary, and document identification means for identifying, based on a result of the reference by the reference means, whether or not the arbitrary document is the particular document.
Here, the reference means may serve also as the verification means, and the document identification means may serve also as the discrimination means.
In data medium handling apparatus of the present invention having the construction described above, from image data of a particular document as a data medium read by the image data reading apparatus and having required information described in a format thereon, document identification information described on the particular document is extracted and the document identification information is registered into the document identification dictionary, and then, image data of an arbitrary document are read by means of the image data reading apparatus, whereafter the image data of the arbitrary document are referred to to detect whether or not the image data of the arbitrary document include the document identification information registered in the document identification dictionary to identify whether or not the arbitrary document is the particular document.
Further, in the data medium handling apparatus of the present invention, from image data of a particular document as a data medium read by the image data reading apparatus and having required information described in a format thereon, document identification information described on the particular document is extracted and the document identification information is registered into the document identification dictionary, and then, image data of the particular document are inputted again and it is verified whether or not the inputted image data of the particular document include the document identification information registered in the document identification dictionary to discriminate whether or not recognition of the particular document is possible, whereafter, when recognition of the particular document is possible, image data of an arbitrary document are read by means of the image data reading apparatus and the image data of the arbitrary document are referred to to detect whether or not the image data of the arbitrary document include the document identification information registered in the document identification dictionary to identify whether or not the arbitrary document is the particular document.
Here, when the document identification information is to be registered into the document identification dictionary, the image data of the particular document read by the image data reading apparatus may be displayed on the display unit and a region which includes an image of the document identification information described on the particular document may be designated from within the image data of the displayed particular document to extract the document identification information.
Or, when the document identification information is to be registered into the document identification dictionary, particular document identification information described on the particular document may be automatically extracted from the image data of the particular document read by the image data reading is apparatus.
Or else, when the document identification information is to be registered into the document identification dictionary, upon registration of the document identification information into the document identification dictionary, the image data of the particular document read upon registration may be stored, and, when the document identification information is to be registered into the document identification dictionary, the stored image data of the particular document may be inputted as image data of the particular document again, and it may be verified whether or not the inputted image data of the particular document include the document identification information registered in the document identification dictionary to discriminate whether or not recognition of the particular document is possible.
Accordingly, the data medium handling apparatus of the present invention is advantageous in that, since a kind of a document whose image data have been read by the image data reading apparatus can be automatically identified, even if a plurality of kinds of documents to be read by the image reading apparatus are present in a mixed condition, an operator can handle the document without being aware of definition of each document and improvement in efficiency in job can be anticipated. The data medium handling apparatus is advantageous also in that an ID number or the like for identification of a document itself need not be described on the document and ordinary documents can be used, and consequently, the data medium handling apparatus can be applied readily to an existing system.
Further, since it can be verified whether or not document identification information designated by the operator is correct, the data medium handling apparatus is advantageous in that erroneous discrimination of a document is eliminated at all and the reliability of the apparatus is improved.
Furthermore, since the reference means serves also as the verification means and the document identification means serves also as the discrimination means, the data medium handling apparatus is advantageous in that the apparatus construction can be simplified.
Further, a data medium handling apparatus according to the present invention is characterized in that it comprises an image reading section for reading an image of a data medium having emblem information, an emblem recognition section for recognizing the emblem image information in the image read by the image reading section and converting the emblem image information into corresponding code information, a display unit for displaying the emblem image information in a condition of a read image, and a read emblem image display control section for causing, when the emblem image information has not successfully been converted into corresponding code information, the unrecognizable emblem image information, whose conversion has not been performed successfully, to be displayed in a condition of a read image on the display unit together with emblem image information associated with the unrecognizable emblem image information, the read emblem image display control section including emphasis display means for emphatically displaying the unrecognizable emblem image information.
Here, the data medium handling apparatus further comprises a data inputting apparatus for inputting correct answer data corresponding to the unrecognizable emblem image information, and correct answer data display means for causing, when the correct answer data are inputted by an operation of the data inputting apparatus, the correct answer data to be displayed in a correct answer data display area different from an unrecognizable emblem image information display area on the display unit.
The data medium handling apparatus may further comprise emphasis display changing means for causing, when correct answer data are inputted by the operation of the data inputting apparatus, the unrecognizable emblem image information which has been displayed in emphasis to be displayed in a different manner of emphasis which signifies that the unrecognizable emblem image information has been modified.
Or, the data medium handling apparatus may further comprise emphasis display changing means for causing, when the correct answer data are displayed in the correct answer data display area different from the unrecognizable emblem image information display area on the display unit by the correct answer data display means, the unrecognizable emblem image information, which has been displayed in emphasis, to be displayed in a different manner of emphasis which signifies that the unrecognizable emblem image information has been modified.
In particular, the data medium handling apparatus according to the present invention wherein an image of a data medium having emblem information is read and emblem image information in the read image is recognized and converted into corresponding code information, is characterized in that, when the emlem image information has not been successfully converted into corresponding code information, the unrecognizable emblem image information which has not been successfully converted is displayed in a condition of a read image on the display unit together with emblem image information associated with the unrecognizable emblem image information, and the unrecognizable emblem image information is displayed in emphasis, and thereafter, when correct answer data corresponding to the unrecognizable emblem image information are inputted by an operation of the data inputting apparatus, the unrecognizable emblem image information which has been displayed in emphasis is displayed in a different manner of emphasis which signifies that the unrecognizable emblem image information has been modified.
In short, a data medium handling apparatus of the present invention is characterized in that it comprises an image reading section for reading an image of a data medium having emblem information, an emblem recognition section for recognizing the emblem image information in the image read by the image reading section and converting the emblem image information into corresponding code information, a display unit for displaying the emblem image information in a condition of a read image, and a read emblem image display control section for causing, when the emblem image information has not successfully been converted into corresponding code information, the unrecognizable emblem image information, whose conversion has not been performed successfully, to be displayed in a condition of a read image on the display unit together with emblem image information associated with the unrecognizable emblem image information, emphasis display means for causing the unrecognizable emblem image information to be displayed in emphasis, a data inputting apparatus for inputting correct answer data corresponding to the unrecognizable emblem image information, emphasis display changing means for causing, when the correct answer data are inputted by an operation of the data inputting apparatus, the unrecognizable emblem image information, which has been displayed emphatically till then, to be displayed in a different manner of emphasis which signifies completion of modification, and recognition information modification means for modifying, when the correct answer data are inputted by the operation of the data inputting apparatus, the unrecognizable emblem information based on the correct answer data.
In the data medium handling apparatus of the present invention having the construction described above, when an image of a data medium having emblem information is read and emblem image information in the read image is recognized and converted into corresponding code information, if the emblem image information has not been successfully converted into corresponding code information, then the unrecognizable emblem image information which has not been successfully converted is displayed in a condition of a read image on the display unit together with emblem image information associated with the unrecognizable emblem image information, and the unrecognizable emblem image information is displayed in emphasis.
Further, when correct answer data corresponding to the unrecognizable emblem image information are inputted by an operation of the data inputting apparatus, the correct answer data are displayed in the correct answer data display area different from the unrecognizable emblem image information display area on the display unit.
Or, when correct answer data corresponding to the unrecognizable emblem image information are inputted by an operation of the data inputting apparatus, the unrecognizable emblem image information which has been displayed in emphasis is displayed in a different manner of emphasis which signifies that the unrecognizable emblem image information has been modified.
Or else, when correct answer data corresponding to the unrecognizable emblem image information are inputted by an operation of the data inputting apparatus, the correct answer data are displayed in the correct answer data display area different from the unrecognizable emblem image information display area on the display unit while the unrecognizable emblem image information which has been displayed in emphasis is displayed in a different manner of emphasis which signifies that the unrecognizable emblem image information has been modified.
Accordingly, with the data medium handling apparatus of the present invention, when the emblem image information has not been successfully converted into corresponding code information, the unrecognizable emblem image information which has not been successfully converted is displayed in a condition of a read image on the display unit together with emblem image information associated with the unrecognizable emblem image information, and the unrecognizable emblem image information is displayed in emphasis. Consequently, the data medium handling apparatus is advantageous in that, even if the emblem image information is unrecognizable because it is, for example, a much deformed character, a confirmation operation with a high degree of accuracy can be performed by referring to image information of preceding and following characters in a train of characters or the like and the labor and the time required for a confirmation operation between correct and wrong can be reduced.
Further, when correct answer data corresponding to the unrecognizable emblem image information are inputted by an operation of the data inputting apparatus, the correct answer data are displayed in the correct answer data display area different from the unrecognizable emblem image information display area on the display unit, and consequently, the emblem image information and the inputted correct answer data can be re-confirmed by visual observation. Consequently, the data medium handling apparatus is advantageous in that the labor and the time required for a confirmation operation between correct and wrong can be reduced and a confirmation operation with a high degree of accuracy can be performed.
Furthermore, when correct answer data corresponding to the unrecognizable emblem image information are inputted by an operation of the data inputting apparatus, the correct answer data are displayed in the correct answer data display area different from the unrecognizable emblem image information display area on the display unit while the unrecognizable emblem image information which has been displayed in emphasis is displayed in a different manner of emphasis which signifies that the unrecognizable emblem image information has been modified. Consequently, the emblem image information and the inputted correct answer data can be re-confirmed by visual observation. Consequently, the data medium handling apparatus is advantageous in that a confirmation operation with a high degree of accuracy can be performed, and identification of whether or not inputting for an unrecognizable emblem or modification to an erroneously recognized emblem has been performed can be made readily, resulting in improvement in visual discernibility to prevent double modifications of the operator and so forth, and besides the labor and the time required for a confirmation operation between correct and wrong can be reduced.
Further, a data medium handling apparatus of the present invention is characterized in that it comprises an image reading section for reading an image of a data medium having emblem information, an emblem recognition section for recognizing the emblem image information in the image read by the image reading section and converting the emblem image information into corresponding code information, a display unit for displaying the emblem image information in a condition of a read image, and a minimum composition unit emblem image information extraction section for extracting emblem image information of a minimum composition unit to be converted into code information in the image for each piece of the emblem image information of the minimum composition unit, and a minimum composition unit emblem image display control section for causing the emblem image information of the minimum composition unit extracted by the minimum composition unit emblem image information extraction section to be displayed in a corresponding relationship to emblem information signified by code information obtained by conversion of the emblem image information.
Here, the minimum composition unit emblem image display control section may have a function of causing a plurality of pieces of the emblem image information of the minimum composition unit of a same kind to be displayed in a corresponding relationship to emblem information signified by code information obtained by conversion of the emblem image information, and include emphasis display means for causing only a desired one of the pieces of emblem image information to be displayed in emphasis.
Or, the data medium handling apparatus may further comprise associated emblem image display means for displaying the one piece of emblem image information to be displayed in a condition of a read image on the display unit together with emblem image information associated with the one piece of emblem image information, and the associated emblem image display means may include emphasis display means for causing the one piece of emblem image information to be displayed in emphasis.
Or else, the data medium handling apparatus may further comprise a data inputting apparatus for inputting correct answer data corresponding to the one piece of emblem image information, and correct answer data display means for causing, when the correct answer data are inputted by an operation of the data inputting apparatus, the correct answer data to be displayed in a correct answer data display area different from an area in which the one piece of emblem image information and emblem image information associated with the one piece of emblem image information are displayed.
Otherwise, the data medium handling apparatus may further comprise a data inputting apparatus for inputting correct answer data corresponding to the one piece of emblem image information, and emphasis display changing means for causing, when correct answer data are inputted by an operation of the data inputting apparatus, the one piece of emblem image information which has been displayed in emphasis to be displayed in a different manner of emphasis which signifies that the one piece of emblem image information has been modified.
Or otherwise, the data medium handling apparatus may further comprise a data inputting apparatus for inputting correct answer data corresponding to the one piece of emblem image information, correct answer data display means for causing, when the correct answer data are inputted by an operation of the data inputting apparatus, the correct answer data to be displayed in a correct answer data display area different from an area in which the one piece of emblem image information and emblem image information associated with the one piece of emblem image information are displayed, and emphasis display changing means for causing, when correct answer data are inputted by the operation of the data inputting apparatus, the one piece of emblem image information which has been displayed in emphasis to be displayed in a different manner of emphasis which signifies that the one piece of emblem image information has been modified.
In particular, according to the present invention, a data medium handling method wherein an image of a data medium having emblem information is read and emblem image information in the read image is recognized and converted into corresponding code information, is characterized in that a plurality of pieces of emblem image information of a minimum composition unit of a same kind to be converted into code information in the image are displayed in a corresponding relationship to emblem information signified by code information obtained by conversion of the emblem image information while only a desired one of the pieces of emblem image information is displayed in emphasis, and then, when correct answer data corresponding to the one piece of emblem image information are inputted by an operation of the data inputting apparatus, the one piece of emblem image information which has been displayed in emphasis is displayed in a different manner of emphasis which signifies that the one piece of emblem image information has been modified, and erroneously recognized information corresponding to the one piece of emblem information is modified based on the correct answer data.
In short, a data medium handling apparatus of the present invention is characterized in that it comprises an image reading section for reading an image of a data medium having emblem information, an emblem recognition section for recognizing the emblem image information in the image read by the image reading section and converting the emblem image information into corresponding code information, a display unit for displaying the emblem image information in a condition of a read image, and a minimum composition unit emblem image information extraction section for extracting emblem image information of a minimum composition unit to be converted into code information in the image for each piece of the emblem image information of the minimum composition unit, a minimum composition unit emblem image display control section for causing a plurality of pieces of the emblem image information of the minimum composition unit extracted by the minimum composition unit emblem image information extraction section to be displayed in a corresponding relationship to emblem information signified by code information obtained by conversion of the emblem image information, emphasis display means for causing only a desired one of pieces of emblem image information to be displayed in emphasis, a data inputting apparatus for inputting correct answer data corresponding to the one piece of emblem image information, emphasis display changing means for causing, when the correct answer data are inputted by an operation of the data inputting apparatus, the one piece of emblem image information, which has been displayed emphatically till then, to be displayed in a different manner of emphasis which signifies completion of modification, and recognition information modification means for modifying, when the correct answer data are inputted by the operation of the data inputting apparatus, erroneously recognized information corresponding to the one piece of emblem image information based on the correct answer data.
In the data medium handling apparatus of the present invention having the construction described above, when an image of a data medium having emblem information is read and emblem image information in the read image is recognized and converted into corresponding code information, emblem image information of a minimum composition unit to be converted into code information in the image is displayed on the display unit in a corresponding relationship to emblem information signified by code information obtained by conversion of the emblem image information.
Where a plurality of pieces of emblem image information of minimum composition units of a same kind are displayed in a corresponding relationship to emblem information signified by code information obtained by conversion of the emblem image information, only a desired one of the pieces of emblem image information is displayed in emphasis.
Or, a desired one piece of emblem image information is displayed in a condition of a read image on the display unit together with emblem image information associated with the piece of emblem image information, and the one piece of emblem image information is displayed in emphasis.
Further, when correct answer data corresponding to the one piece of emblem image information are inputted by an operation of the data inputting apparatus, the correct answer data are displayed in the correct answer data display area different from the area in which the one piece of emblem image information and emblem image information associated with the one piece of emblem image information are displayed.
Or, when correct answer data corresponding to the one piece of emblem image information are inputted by an operation of the data inputting apparatus, the one piece of emblem image information which has been displayed in emphasis is displayed in a different manner of emphasis which signifies that the one piece of emblem image information has been modified.
Or else, when correct answer data corresponding to the one piece of emblem image information are inputted by an operation of the data inputting apparatus, the correct answer data are displayed in the correct answer data display area different from the area in which the one piece of emblem image information and emblem image information associated with the one piece of emblem image information are displayed, and the one piece of emblem image information which has been displayed in emphasis is displayed in a different manner of emphasis which signifies that the one piece of emblem image information has been modified.
Accordingly, with the data medium handling apparatus of the present invention, since emblems are displayed in a unit which allows them to be visually observed readily in such a manner that only same emblems are displayed in a same area of the display unit and only a desired one of pieces of emblem image information is displayed in emphasis, an operator can confirm whether a result of recognition is correct or wrong only by referring to image information without referring to associated image information, thereby decreasing movements of the eye of the operator. Further, only emblem image information being processed is displayed in emphasis to decrease the number of objects of comparison. Consequently, the data medium handling apparatus is advantageous in that the labor and the time required for a confirmation operation between correct and wrong can be reduced remarkably.
Further, since a desired one piece of emblem image information is displayed in a condition of a read image on the display unit together with emblem image information associated with the piece of emblem image information and the one piece of emblem image information is displayed in emphasis, the data medium handling apparatus is advantageous in that, even if the emblem image information is unrecognizable because it is, for example, a much deformed character, a confirmation operation with a high degree of accuracy can be performed by referring to image information of preceding and following characters in a train of characters or the like and the labor and the time required for a confirmation operation between correct and wrong can be reduced.
Further, when correct answer data corresponding to the one piece of emblem image information are inputted by an operation of the data inputting apparatus, the correct answer data are displayed in the correct answer data display area different from the area in which the one piece of emblem image information and emblem image information associated with the one piece of emblem image information are displayed, and the one piece of emblem image information which has been displayed in emphasis is displayed in a different manner of emphasis which signifies that the one piece of emblem image information has been modified. Consequently, the emblem image information and the inputted correct answer data can be re-confirmed by visual observation. Consequently, the data medium handling apparatus is advantageous in that a confirmation operation with a high degree of accuracy can be performed, and identification of whether or not inputting for an unrecognizable emblem or modification to an erroneously recognized emblem has been performed can be made readily, resulting in improvement in visual discernibility to prevent double modifications of the operator and so forth, and besides the labor and the time required for a confirmation operation between correct and wrong can be reduced.
Further, a data medium handling apparatus of the present invention is characterized in that it comprises an image data inputting apparatus for inputting image data of a data medium including information of a plurality of items, a display unit having an image data display area and an item display area which has a plurality of item data inputting display area portions, a display control apparatus for causing the image data obtained by the image data inputting apparatus to be displayed in the image data display area and for causing, when data corresponding to an image data portion of a desired one of the items are to be inputted by a data inputting means, a cursor indication to be displayed at a desired one of the item data inputting display area portions in the item display area, and image data partial emphasis display means for causing, when the cursor indication is displayed at the desired one of the item data inputting display area portions in the item display area by the display control apparatus in a condition wherein the image data including the information of the plurality of items are displayed in the image data display area of the display unit, only the corresponding image data portion of the desired item to be displayed in emphasis.
Here, the display control apparatus may include, in addition to the image data partial emphasis display means, image data emphasis display cancellation means for canceling, when the cursor indication is moved to another one of the item data inputting display area portions different from the desired item data inputting display area portion by cursor movement operation means, the emphasis display of the image data portion corresponding to the item, and emphasis display transition control means for controlling the image data partial emphasis display means so that only the image data portion of the different item corresponding to the different item data inputting display area portion may be displayed in emphasis in response to the image data emphasis display cancellation means.
Or, the display control apparatus may include, in addition to the image data partial emphasis display means, image data emphasis display cancellation means and emphasis display transition control means, display screen scrolling means for causing a portion of the image data, which is not displayed on the display unit, to be displayed on the display unit.
Further, the image data partial emphasis display means may include changing means capable of changing the emphatically displayed portion in response to an instruction from the outside.
In the data medium handing apparatus of the present invention having the construction described above, when image data of a data medium which includes information of a plurality of items are displayed on the display unit, only a portion of the image data corresponding to a certain one of the items is displayed in emphasis.
When data corresponding to an image data portion of a desired one of items are to be inputted in a condition wherein image data of a data medium including information of a plurality of items are displayed in the image data display area on the display unit, if the cursor indication is displayed at a desired one of the item data inputting display area portions of the item display area set separately from the image data display area on the display unit, then only the image data portion of the corresponding desired item is displayed in emphasis, and, when data corresponding to an image data portion of another one of the items are to be thereafter inputted, if the cursor indication is moved from the desired item data inputting display area portion to a different one of the item data inputting display area portions, then the emphasis display of the corresponding image data portion of the desired item is canceled and the corresponding image data portion of the different item is displayed in emphasis.
Or, when data corresponding to an image data portion of a desired one of items displayed on the display unit are to be inputted in a condition wherein part of image data of a data medium including information of a plurality of items are displayed in the image data display area on the display unit, if the cursor indication is displayed at a desired one of item data inputting display area portions of the item display area set separately from the image data display area on the display unit, then only the image data portion of the corresponding desired item is displayed in emphasis, and, when data corresponding to an image data portion of a different one of the items which is not displayed on the display unit are to be thereafter inputted, if the cursor indication is moved from the desired item data inputting display area portion to a different one of the item data inputting display area portions, then the emphasis display of the corresponding image data portion of the desired item is canceled while scrolling is performed so that the image data portion of the different item may be be displayed in the image data display area of the display unit to display the image data portion of the different item in the image data display area of the display unit and the corresponding image data portion of the different item is displayed in emphasis.
Accordingly, with the data medium handling apparatus of the present invention, if the cursor indication is displayed at one of the item data inputting display area portions to which data are to be inputted, then the image data portion of an item to be referred to simultaneously is displayed in emphasis. Consequently, even if an operator first removes the eye thereof from the display unit and then watches the display unit again, the operator is prevented from observing a wrong item. Consequently, the data medium handling apparatus is advantageous in that erroneous recognition in matching when image data are referred to is prevented thereby to raise the visual observability and the labor and the time required for an inputting operation can be prevented while erroneous inputting is prevented.
Further, when image data of an item to be referred to are not displayed on the display unit, automatic scrolling can be performed in response to movement of the emphasis display portion. Consequently, the data medium handling apparatus is advantageous in that a wasteful operation such as, for example, depression of a key can be reduced.
Furthermore, it is possible also for the operator to change the kind or the region of the emphasis display portion. Consequently, the data medium handling apparatus is advantageous in that it can be used in accordance with an application.